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Interpreting Local Surveillance Data on Cancer Screening Participation 

The Behavioral Risk Factor Surveillance System (BRFSS) is a collaborative effort between 
the Centers for Disease Control and Prevention (CDC) and states to monitor selected 
behaviors and health indicators across the country. 1 It is based on anonymous telephone 
interviews of adult participants contacted by random-digit dialing. BRFSS was originally 
developed to provide state-level data but there has been increasing demand for local data. 

In 2002, CDC began releasing Selected Metropolitan/Micropolitan Area Risk Trends 
(SMART) BRFSS prevalence estimates for some topics, including cancer screening 
participation. 2 SMART estimates are available for some Metropolitan or Micropolitan 
Statistical Areas (MSAs) used by the US Census Bureau. Micropolitan statistical areas 
contain at least one urban area with a population between 10,000 and 49,999; metropolitan 
statistical areas contain at least one urban area with a population of 50,000 or greater. To 
be included in CDC SMART BRFSS reporting, an MSA must have at least 500 completed 
interviews and must meet certain other numeric criteria in order to provide statistically 
reliable estimates. 

Four areas in Montana were included in SMART BRFSS reporting in 2006: the Billings 
Metropolitan Statistical Area (comprised of Yellowstone and Carbon Counties), the Great 
Falls Metropolitan Statistical Area (Cascade County), the Missoula Metropolitan Statistical 
Area (Missoula County) and the Kalispell Micropolitan Statistical Area (Flathead County). 
These are named for their major cities but county-level estimates may also be available for 
these regions. Only the Billings MSA has SMART BRFSS reports for prior years (2004 and 
2005, but cancer screening participation questions have only been asked in even years 
since 2000). 

The local prevalence estimates provided by SMART BRFSS must be interpreted with 
caution. CDC provides the following guidelines: 2 

• In comparing SMART areas to the state as a whole, to other areas, or across years 
within an area, it is necessary to use Confidence Intervals (CI) to evaluate 
differences between prevalence estimates. A CI measures the uncertainly around 
an estimate. If CIs overlap, the estimates should not be interpreted as different, 
even though they may look quite different. 

• SMART sample sizes are too small to support reliable analysis by categories such 
as sex, age group, race, education, or income. Many behaviors, including cancer 
screening participation, vary by these factors. The populations within SMART 



http://www.cdc.gov/brfss/ 
http://apps.nccd.cdc.gov/brfss-smart/index.asp 
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geographic units may also vary by these factors. 

• SMART estimates are not available if the total number of respondents for a question 
(denominator) is less than 50. Estimates should be evaluated carefully if the number 
of respondents in a cell (numerator) is less than 50. 

• A SMART estimate is not available if the CI is greater than + 10. 

Our surveillance data are based on projecting statewide prevalence estimates from samples 
of participants whom we hope represent the total population accurately. On the whole, 
random-digit dialing and a procedure called weighting (assigning each participant a number 
or statistical weight for analysis to reflect the fact that he or she represent many other 
similar individuals based on sex, age, race, and county of residence) results in a sample 
that resembles the state population fairly closely. Nevertheless, there is an element of 
uncertainty arising from this process, and this uncertainty is measured by the CI around the 
prevalence estimate. It is conventional to use a 95% CI, indicating that the true value of the 
prevalence is within that interval with 95% certainty. Other CIs (e.g., 90%, 99%) are used 
less often. 

The graphs and tables on the facing page present estimated colorectal and breast cancer 
screening participation among age-eligible Montana adults statewide and in the four 
SMART regions in 2006. Although the prevalence estimates for ever having a 
sigmoidoscopy or colonoscopy range from 52.7% to 61.5%, none can be considered 
different from any of the others because all the CIs overlap. Participation rates are not 
statistically different for mammography in spite of the apparently large difference between 
the extreme values of 73.8% and 83.0%, because the CIs for all the estimates overlap. 

If CIs overlap, the estimates should not be described as different. It is common to see 
descriptions such as "screening participation is higher in Community X than elsewhere 
although the difference does not achieve the .05 level of statistical significance." This may 
be done to focus on the apparent difference between estimates rather than on the likelihood 
that the difference is real. 

In general, the smaller the sample size, the larger the CI around a prevalence estimate. In 
these examples, there were at least 500 total respondents from each SMART MSA, but 
there were only 198 to 302 age-eligible adults for the colonoscopy question in each MSA, 
and only 160 to 185 age-eligible women for the mammography question. The CIs around 
the estimates were + 6.0% or greater for colonoscopy for the MSAs and + 5.5% or greater 
for mammography. It is not feasible to increase the local sample sizes enough to achieve 
more precise estimates of local screening participation, although in certain circumstances it 
may be possible to combine several years of data, provided prevalence rates have not 
changed rapidly over time. For most of the cancer screening participation questions, 
prevalence rates statewide have been fairly stable for several survey cycles. 






Montana Cancer Control Section 



Ever Had Sigmoidoscopy or Colonoscopy 
Adults Age 50 and Older, Montana BRFSS 2006 
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Ever Had Sigmoidoscopy or Colonoscopy Adults Age 50 and Older, Montana BRFSS 2006 




Estimate, % 


95% Confidence Interval 


Statewide 


52.9 


50.9 - 54.9 


Billings MSA 


55.3 


51.7 - 65.0 


Great Falls MSA 


60.9 


54.8 - 66.9 


Kalispell MSA 


52.7 


46.0 - 59.3 


Missoula MSA 


61.5 


54.8-68.1 



Had Mammogram in Past Two Years 
Women Age 50 and Older, Montana BRFSS 2006 
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Had Mammogram Within Past Two Years, Women Age 50 and Older, Montana BRFSS 2006 




Estimate, % 


95% Confidence Interval 


Statewide 


76.6 


74.4 - 78.8 


Billings MSA 


73.8 


66.1-81.4 


Great Falls MSA 


83.0 


77.5 - 88.4 


Kalispell MSA 


81.0 


74.9 - 87.0 


Missoula MSA 


80.0 


72.9-87.1 
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The table below shows the effect of sample size on CIs around mammography prevalence 
for the statewide BRFSS from 1995 through 2006, as the total number of participants in 
BRFSS increased from approximately 1,200 to more than 6,000 and the number of age- 
eligible women increased from 227 to 1,990. It is possible to identify a trend of increasing 
mammography participation across the 12 years although, with the exception of 1999 and 
2000, no pairs of adjacent years are statistically significantly different. 



The Effect of Sample Size on the Precision of Estimated Prevalence of Mammography 

Montana BRFSS 1995 to 2006 




1995 


1996 


1997 


1998 


1999 


2000 


2002 


2004 


2006 


Sample size 


277 


423 


450 


446 


480 


790 


1,095 


1,553 


1,990 


Prevalence, % 


63.8 


68.8 


70.4 


72.3 


71.2 


79.3 


76.2 


77.7 


76.6 


Confidence Interval, + % 


6.2 


4.8 


4.4 


4.3 


4.2 


3.3 


3.2 


2.5 


2.2 



The number of respondents almost doubled between 1995 and 1999, reducing the 
Confidence Interval from + 6.2% to + 4.2%. Between 1999 and 2006, the number of 
respondents quadrupled, reducing the Confidence Interval to + 2.2%. This reflects 
excellent precision for the statewide prevalence estimates in recent years, but also 
illustrates the large numbers that would be required to increase the precision of local 
estimates. 



Please visit our website at www.cancer.mt.gov 

For more information about the Montana Cancer Control Program, contact Ginny Furshong, Program Manager, 406- 
444-6888, qfurshonq@mt.gov 

For more information about the Montana Breast and Cervical Health Program, contact Karan Kunz, Program 
Manager, 406-444-0063, kkunz@mt.gov 

For more information about the Montana Central Tumor Registry, contact Debbi Lemons, Program Manager, 406- 
444-2618, dlemons@mt.gov 

For more information about cancer data and analysis, contact Carol Ballew, PhD, Epidemiologist, 406-444-6988, 
cballew@mt.gov 

For more information about the Montana BRFSS, contact the Montana BRFSS Office, Joanne Oreskovich, PhD, 
BRFSS Director/Epidemiologist at 406-444-2973, ioreskovich@mt.gov , or Susan Cummings, BRFSS Coordinator, 406- 
444-4111, scumminqs@mt.gov 

2,500 copies of this document were produced at a cost of $0.00 per copy, for a total cost of $0000.00 for printing and $0 
for distribution. 



Alternative formats of this document will be provided upon request. Please contact Dr. Ballew. 

Montana Cancer Control Program 

Montana Department of Health and Human Services 

1400 Broadway C-317, PO Box 202951 

Helena, MT 59620-2951 
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